Road traffic death coding quality in the WHO Mortality Database

Abstract Objective To evaluate the precision and dependability of road traffic mortality data recorded in the World Health Organization Mortality Database and investigate how uncorrected data influence vital mortality statistics used in traffic safety programmes worldwide. Methods We assessed country and territory-specific data quality from 2015 to 2020 by calculating the proportions of five types of nonspecific cause of death codes related to road traffic mortality. We compared age-adjusted road traffic mortality and changes in the average annual mortality rate before and after correcting the deaths with nonspecific codes. We generated road traffic mortality projections with both corrected and uncorrected codes, and redistributed the data using the proportionate method. Findings We analysed data from 124 countries and territories with at least one year of mortality data from 2015 to 2020. The number of countries and territories reporting more than 20% of deaths with ill-defined or unknown cause was 2; countries reporting injury deaths with undetermined intent was 3; countries reporting unspecified unintentional injury deaths was 21; countries reporting unspecified transport crash deaths was 3; and countries reporting unspecified unintentional road traffic deaths was 30. After redistributing deaths with nonspecific codes, road traffic mortality changed by greater than 50% in 7% (5/73) to 18% (9/51) of countries and territories. Conclusion Nonspecific codes led to inaccurate mortality estimates in many countries. We recommend that injury researchers and policy-makers acknowledge the potential pitfalls of relying on raw or uncorrected road traffic mortality data and instead use corrected data to ensure more accurate estimates when improving road traffic safety programmes.


Introduction
Globally, road traffic crashes cause approximately 1.3 million preventable deaths and 50 million injuries each year. 1 In 2021, the United Nations (UN) General Assembly resolution 74/299 committed to reduce by 50% the number of deaths and injuries caused by road traffic crashes worldwide by 2030. 2 Accurate and reliable mortality data serve as the foundation for tracking advancements and forecasting future progress in attaining the objectives of global road traffic safety initiatives.Among available data sets for this purpose, arguably the most comprehensive is the World Health Organization (WHO) Mortality Database, which compiles yearly mortality data reported by Member States from their civil registration and vital statistics systems. 35][6][7] One limitation of the WHO Mortality Database is that it includes a substantial number of nonspecific death codes that do not comply with the principles of the International Statistical Classification of Diseases and Related Health Problems, 10 th revision (ICD-10). 8Despite this notable limitation, raw and uncorrected data is still being used by researchers for analysis. 9,10fforts have been made by the Global Health Estimates study group to correct data quality challenges arising from incorrect death codes. 5Their research showed that data users can provide comprehensive and comparable cause of death estimates by assessing and redistributing nonspecific ICD-10 codes. 5,11Unfortunately, the Global Health Estimates study group did not publish a detailed quality analysis of raw road traffic mortality data, nor did they quantify its influence.
The GBD study group also assessed and redistributed nonspecific ICD-10 codes for 204 countries and territories between 1990-2019.However, this team did not publicly release information on the coding quality of the mortality data they used, nor did they publish a comprehensive assessment of the quality of that data. 6 few researchers have used related data sets from specific Member States, namely Brazil and South Africa, to explore the impact of death coding quality.[12][13][14][15] However, we did not find any research which systematically analysed the quality of road traffic mortality data from the WHO Mortality Database, or quantified its influence on monitoring progress towards global health goals or projecting future global road traffic safety trends.
The main goal of this study is to assess the availability and coding quality of road traffic mortality data from the WHO Mortality Database, using data from the years 2015 to 2020.Additionally, we seek to investigate the influence of coding quality on data used to monitor the progress of global road traffic safety initiatives.

Indicators for data quality
][19][20] To assess data availability, we used the presence or absence of road traffic mortality data in the WHO Mortality Database to reflect on overall data availability.We quantified data availability in number of years since 2015 by determining whether mortality data were available for 0 years (unavailable), 1 year, 2 years, 3 years, 4 years, 5 years or 6 years.

Redistribution of deaths
We used the proportionate method 21 to redistribute deaths with nonspecific codes to cause-specific codes.This method assumes that deaths with nonspecific codes follow the same all-cause distribution as deaths with cause-specific codes, and can therefore be redistributed to all cause-specific deaths.Furthermore, the proportionate method is preferable when there is limited covariate data available for use. 22revious studies found that corrected results for road injuries, obtained through the proportionate method, were more accurate in approximating true values compared to the use of more intricate methods. 19

Statistical analysis
According to the ICD-10 manual, 8 the first four types of nonspecific ICD-10 codes tend to underestimate overall road traffic mortality and user-specific road traffic mortality, while the last type, unspecified unintentional road traffic deaths, tends to specifically underestimate road traffic mortality for occupants and motorcyclists.We therefore redistributed deaths with the first four types of nonspecific ICD-10 codes to calculate the age-adjusted overall road traffic mortality and user-specific road traffic mortality for pedal cyclists and pedestrians, and used deaths with all five types of nonspecific codes to calculate the age-adjusted road traffic mortality for occupants and motorcyclists (Fig. 1).When the proportion of deaths  Coding quality of road traffic mortality data Junjie Hua et al.
with nonspecific codes was < 30%, 30%-49%, 50%-69% or ≥ 70% in a particular year, we used the proportion of cause-specific deaths in the same year (the study year); a three-year period (the preceding year, the study year, and the next year); a five-year period (two preceding years, the study year, and the next two years); or a seven-year period (three preceding years, the study year, and the next three years) to redistribute deaths with nonspecific codes.When relevant data for the preceding and/ or following years were unavailable, we used all available data to compute corrections.
To assess the influence of deaths with nonspecific codes on overall road safety data, we compared age-adjusted overall road traffic mortality rates and user-specific road traffic mortality rates for occupants, motorcyclists, pedal cyclists and pedestrians, both before and after correcting (redistributing) deaths with nonspecific ICD-10 codes.Age-adjusted mortality rates were calculated using the new WHO world standard population values. 23The ratio of corrected/uncorrected mortality rates was then used to quantify the proportionate influence of deaths with nonspecific codes on road safety statistics.
To assess the impact of nonspecific ICD-10 codes on trends in road traffic crash data, we compared the average annual change rate for road traffic deaths before and after correcting and redistributing deaths with nonspecific codes.Based on the robustness to extreme values reported in the previous analysis, 24 we selected the geometric mean method to calculate the average annual change rate of road traffic mortalities.In addition, we used the average annual change rate of road traffic deaths  Notes: We obtained data from the WHO Mortality Database.The ICD-10 codes for each type are: R95, R96, R98 and R99 for deaths with ill-defined or unknown cause; Y34, Y87.2 and Y89.9 for injury deaths with undetermined intent; X59 for unspecified unintentional injury deaths; V99 and Y85.9 for unspecified transport crash deaths; and V87.0-V87.8 and V89.2 for unspecified unintentional road traffic deaths.The number of countries or territories reporting data changed each year and was consistently fewer than the total count of 97 countries or territories that reported data at some point between 2015 and 2020.

Data availability
As of 27 February 2023, 124 countries and territories reporting to the WHO Mortality Database had at least one complete year of mortality data available between 2015 and 2020 (Fig. 2).The WHO European Region had the most years of available data (248 years), while the South-East Asia Region had the fewest (10 years; Table 1).

Coding quality
Out of the 97 countries and territories using the 4-digit ICD-10 coding system for reporting cause of death, the average number of countries and territories per year reporting over 20% nonspecific mortalities for each type of nonspecific code were: (i) two countries and territories had codes of illdefined and unknown cause; (ii) three countries and territories had codes of injury deaths with undetermined intent; (iii) 21 countries and territories had codes of unspecified unintentional injury deaths; (iv) three countries and territories had codes of unspecified transport crash deaths; and (v) 30 countries and territories had codes of unspecified unintentional road traffic deaths (Fig. 3).

Impact of coding quality
After correcting for mortality data with nonspecific ICD-10 codes, shifts were observed in age-adjusted overall and user-specific road traffic mortality for most of the 93 countries and territories with available data.Notably, over the 6-year period from 2015 to 2020, on average seven countries and territories experienced a greater than 50% increase in age-adjusted overall road traffic mortality (Fig. 4).
For occupants, an average of 27 countries and territories per year experienced a greater than 50% increase in age-adjusted mortality.For motorcyclists, an average of 27 countries and territories experienced a greater than 50% increase in age-adjusted mortality.For pedal cyclists, only six countries and territories on average saw a greater than 50% increase in age-adjusted mortality.Lastly, for pedestrians, nine countries and territories on average experienced a greater than 50% increase in age-adjusted mortality.
We calculated the average annual change rate for the 76 countries and territories presenting complete mortality data for two or more years between 2015 and 2020.Before correction, the average annual change rate of road traffic deaths was greater than zero (positive) for 16 countries and territories; at zero (neutral) for two territories; and ICD: International statistical classification of diseases and related health problems, 10th revision.Note: The number of countries or territories reporting data changed each year and was consistently fewer than the total count of 93 countries or territories that reported data at some point between 2015 and 2020.Bull World Health Organ 2023;101:637-648| doi: http://dx.doi.org/10.2471/BLT.23 Before correcting mortality data with nonspecific ICD-10 codes, the number of road traffic deaths was projected to increase in 16 (21%) countries and territories, remain unchanged in two (3%) territories, and decrease in 58 (76%) countries and territories.In contrast, the number of road traffic deaths was projected to increase in 22 (29%) countries and territories, and decrease in 54 (71%) countries and territories after correction.
Prior to correction, 22 countries or territories also appeared to be on track to achieve global safety targets set for the Global plan for the decade of action for road safety 2021-2030. 2However, after correction, Republic of Moldova and Türkiye shifted from the 'achieve the target' category to the 'unable to achieve the target' category.Additionally, the projected percentage change in road traffic mortality rates between 2021 and 2030 underwent significant alterations of more than 50% for four countries and territories: Lebanon (changing from 4904.7% to 1373.9%);Thailand (changing from 194.8% to 48.7%); Tunisia (changing from −35.5% to 97.1%); and El Salvador (changing from −20.4% to 44.8%; Table 3).

Discussion
Our analysis generated three main sets of findings.First, the number of countries and territories reporting mortality data to WHO for the 2015-2020 period was low, with some WHO regions underrepresented.Previous studies reported that mortality data were available for at least one year in the WHO Mortality Database for 115 countries and territories from 1950 to 2003, 25 and for 83 countries and territories from 2000 to 2009. 18Our findings suggest that the availability of Coding quality of road traffic mortality data Junjie Hua et al.
mortality data in the WHO Mortality Database has not improved over the period from 2015 to 2020, as we only observed 52 countries and territories reporting complete data for all 6 years.Possible reasons for poor data availability include: (i) some countries and territories have not established reliable vital statistics systems to collect mortality data; 26 (ii) vital registration systems in some countries and territories are disrupted by war or political unrest; 27,28 (iii) several countries and territories do not use standard ICD codes to record mortality data; 29 (iv) some countries and territories lack adequate resources and qualified coders to reliably complete death certificates and gather mortality data; 30,31 and (v) some countries and territories refuse to submit their data to WHO, perhaps due to concerns about data misinterpretation or misuse, intentional or unintentional privacy disclosure, or loss of data ownership. 32,33econd, about three fifths of countries and territories reported more than 20% of nonspecific ICD-10 codes for road traffic deaths, particularly for deaths coded with unspecified unintentional injury deaths and unspecified unintentional road traffic deaths.This unsatisfactory coding quality of data corroborates the results of a previous study, 18 which highlighted the link between less cause specificity and the proportion of unspecified unintentional road injury deaths across six types of ICD codes in WHO mortality data, as updated on 21 April 2009.
Third, we found that nonspecific ICD-10 codes underestimated road traffic mortality rates by more than 20% for 70 countries and territories, influencing our ability to accurately monitor progress and project future outcomes for global road safety development targets.These findings are generally consistent with previous reports for individual countries like Mexico, 34 Republic of Korea 35 and the USA, 36   Coding quality of road traffic mortality data Junjie Hua et al.
ally, the projected future rate changes changed from 'achieve the target' to 'unable to achieve the target' status for two countries.Irrespective of the negative outcomes noted above, coding quality has improved in many countries and territories in recent years; despite a few exceptions (for example, Brazil and the United Republic of Tanzania) where coding quality has declined due to insufficient numbers of certified medical coders and/or increased workloads per coder. 37,38ur findings have several policy implications.First, to enhance the WHO Mortality Database's data availability and its significance in decision-making, policy-making and scientific research, Member States possessing mortality data need to prioritize data-sharing for national and global health.These Member States should promptly report the data to WHO as required.For those without reliable mortality data, implementing the WHO civil registration and vital statistics strategic implementation plan 2021-2025 will help improve civil registration and vital statistics capacity, and the SCORE for Health Data Technical Package could be used as a technical tool in these countries. 39,40econd, we propose a series of measures to enhance coding quality in countries and territories facing poor quality death coding.These actions include standard training of death certificate coders based on standards from the Data for Health Initiative. 41Additionally, using procedures outlined in the Analysis of National Causes of Death for Action tool can improve the accuracy of data coding. 42urthermore, the development of artificial intelligence-driven automatic coding tools with high predictive performance may help improve the quality of coding by overcoming shortages of qualified death certificate coders. 43or example, deep semantic matching or classification models based on analogical reasoning 44 and federated learning 45 could automatically code deaths, with human coders only used to confirm artificial intelligenceselected cases.Resources should be provided by international donors to countries and territories which cannot afford to implement these changes, or that understandably prioritize other national efforts.(continues. ..)

Country or territory, by WHO region
Projected changes in road traffic mortality, % Coding quality of road traffic mortality data Junjie Hua et al.
Lastly, injury researchers and policy-makers should exercise caution while using raw mortality data, as recommended by WHO on its database website. 3More emphasis should be placed on rigorously evaluating data quality, making critical corrections to raw data, and interpreting the results with careful consideration.This study has several limitations.First, we focused only on nonspecific mortality coding and were unable to validate other quality problems such as underreporting, overreporting or misclassification across all death causes.Thus, our findings do not necessarily reflect the importance of all data quality problems on statistical interpretation.Second, we were unable to investigate factors influencing coding quality in each country and territory due to the absence of relevant policy documents outlining data collection strategies.Setting up a large-scale research programme would be beneficial to explore these factors and develop viable solutions to inconsistencies in the data.Last, the proportionate method we used to redistribute mortalities with nonspecific codes relies on the assumption that deaths with nonspecific ICD-10 codes follow the same cause distribution pattern as deaths with specific codes. 21f this assumption is violated, our corrected results may be invalid.
In conclusion, as injury researchers and policy-makers, it is imperative that we acknowledge the potential pitfalls of relying on raw or uncorrected road traffic mortality data and approach analytical findings with utmost caution.Only through rigorous assessment and interpretation can we understand the complexities of road traffic safety data and make informed decisions to improve global road safety.■ Bull World Health Organ 2023;101:637-648| doi: http://dx.doi.org/10.2471/BLT.23

Fig. 1 .
Fig. 1.Categorization and distribution of deaths with nonspecific ICD-10 codes for assessing coding quality of road traffic mortality data

Fig. 2 .
Fig. 2. Overall availability of road traffic mortality data in the WHO Mortality Database, 2015-2020

Table 1 . Years of available road traffic mortality data from six WHO regions in the WHO Mortality Database, 2015-2020 WHO Region No of countries or territories Years of available data
WHO: World Health Organization.

Résumé Qualité des codes attribués aux accidents mortels de la circulation dans la base de données de l'OMS sur la mortalité Objectif Déterminer
le niveau de précision et de fiabilité des informations concernant le nombre de décès liés aux accidents de la route enregistrées dans la base de données de l'Organisation mondiale de la Santé sur la mortalité, mais aussi étudier l'influence des données non corrigées sur les statistiques de mortalité, qui jouent un rôle essentiel dans les programmes de sécurité routière à travers le monde.